Buffer management for optimized processing in media pipeline

ABSTRACT

Automated processes, computing systems, computing devices, and other aspects of a data processing system improve reliability in transmitting digital media content over a network using resource constrained hardware. Media content may be received from a media source and used to generate data segments. A reference to a first segment of the media content stored in the buffer is written to a message queue. A computing device switches into copy data mode in response to the number of references in the buffer being greater than or equal to a first threshold value. A second segment of the media content is written directly to the message queue.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Pat. Application No. 17/741,255filed on May 10, 2022, and entitled “BUFFER MANAGEMENT FOR OPTIMIZEDPROCESSING IN MEDIA PIPELINE,” which is incorporated herein byreference.

TECHNICAL FIELD

The following discussion generally relates to media signal transmission.Various embodiments may be used in connection with media players,placeshifting devices, digital video recorder (DVR) devices, video gameplayers and/or any other devices that transmit or receive streamingmedia or other digital content via the Internet or a similar network.

BACKGROUND

In the past, television viewing typically occurred at home, with one ormore family members gathered in front of a television to watch abroadcast program. While conventional television viewing continues to bea popular activity, media streaming is becoming increasinglycommonplace. Many viewers now watch their television and other mediacontent as media streams delivered to phones, tablets, portablecomputers, or other mobile devices. Other devices such as video gameplayers, digital video recorders, television receivers, set top boxesand other media devices are becoming increasingly capable of handlingstreaming media content, and these have been well-received by viewers.

As viewers enjoy their content on a variety of playback devices, theirexpectations grow regarding reliability and speed of media streamsreceived on different devices over a wide variety of networks.Placeshifting devices at homes can present particular challenges due totheir typically limited technical capabilities. Placeshifting devicesinstalled in user homes may have relatively low processing power,caching capacity, and memory speed for encoding and transmitting mediato user devices, which can present challenges. For example, processingor memory limitations may cause some media sources to lose data when aburst output is generated or when media data is otherwise receivedfaster than it is encoded or transmitted. These conditions can lead toundesirable data loss.

It is therefore desirable to create devices, systems, and automatedprocesses to transmit streaming media content from devices with limitedtechnical capabilities to client devices operating on the Internet orother wide area networks. It is particularly desirable to transmit mediacontent from devices having limited processing and caching capabilitieswithout losing data during burst conditions or other adverse conditions.Other desirable features and characteristics will become apparent fromthe subsequent detailed description and the appended claims, taken inconjunction with the accompanying drawings and this background section.

BRIEF DESCRIPTION

Various embodiments relate to different automated processes, computingsystems, devices, and other aspects of a data processing system that isexecuted by a processor of a server device to ingest media content witha buffer using pointers instead of block copies when resources areavailable. The automated process includes receiving media content from amedia source and generating data segments from the media content. Afirst address referencing a first segment of the media content stored inthe buffer is written to a message queue. The process checks whether anumber of addresses stored in a message queue is greater than or equalto a first threshold value. A second segment of the media content iswritten to the message queue in response to the number of addressesstored in the message queue being greater than or equal to the firstthreshold value.

Some embodiments provide an automated process performed by a serverdevice wherein the first threshold value may be equal to a maximumlength of the buffer. The maximum length of the buffer is equal to amaximum number of data segments that fit into the buffer. The datasegments comprise multi-bitrate segments. The buffer is stored in acache-level memory of the server device. The message queue is stored inmain memory of the server device, with main memory having slower readand write speeds than the cache-level memory. The server device checkswhether the number of addresses stored in the message queue is less thanor equal to a second threshold value. The server device writes to themessage queue a second address referencing a third segment of the mediacontent stored in the buffer in response to the number of addressesstored in the message queue being less than or equal to the secondthreshold value. The second threshold value may be 1 in this example.

In various embodiments, an automated process executed by a processor ofa server device ingests media content. The automated process includesthe steps of receiving the media content from a media source, preparingsegments of the media content for consumption using a first thread, andwriting to a message queue an address of a first segment from thesegments stored in a buffer using the first thread. The first threadwrites the address in response to the processor being in a copyreference mode. The first thread switches into copy data mode inresponse to detecting a first switching condition of the message queue.The first thread writes a second segment of the media content to themessage queue in response to the processor being in the copy data mode.The first thread switches into the copy reference mode in response todetecting a second switching condition of the message queue.

In this example, the automated process detects the second switchingcondition of the message queue by checking whether the number ofaddresses stored in the message queue is greater than or equal to athreshold value. The first thread writes to the message queue a secondaddress referencing a third segment of the media content stored in thebuffer in response to the number of addresses stored in the messagequeue being greater than or equal to the threshold value. The thresholdvalue is a maximum length of the buffer. The segments can includemulti-bitrate segments. The buffer is stored in a cache-level memory ofthe server, and the message queue is in main memory of the serverdevice. Main memory typically has slower read and write speeds than thecache-level memory. Detecting the second switching condition of themessage queue includes checking whether the number of addresses storedin the message queue is less than or equal to a threshold value. Asecond address referencing a third segment of the media content storedin the buffer is written to the message queue in response to the numberof addresses stored in the message queue being less than or equal to thethreshold value. A second thread consumes the address of the firstsegment and read the first segment from the buffer to encode the mediacontent. The message queue comprises a first-in-first-out queue.

Other embodiments could relate to a server device comprising aprocessor, a non-transitory data storage, and an interface to a network,wherein the non-transitory data storage is configured to storecomputer-executable instructions that, when executed by the processor,perform an automated process to ingest media content for transmission onthe network. The automated process includes receiving the media contentfrom a media source, generating a plurality of data segments from themedia content, and writing a first address to a message queue. The firstaddress references a first segment of the media content stored in abuffer. The process includes checking whether a number of addressesstored in the message queue is greater than or equal to a firstthreshold value. The processor switches into a copy data mode inresponse to the number of addresses stored in the message queue beinggreater than or equal to the first threshold value. Other devices,systems, and automated processes may be formulated in addition to thosedescribed in this brief summary section.

DRAWINGS

FIG. 1 illustrates an example of a system for transmitting streamingmedia content from a server device to a client device on a network.

FIG. 2A illustrates an example of a circular buffer with data segmentsreferenced by addresses stored in a messaging queue.

FIG. 2B illustrates an example of a full circular buffer with datasegments referenced by addresses stored in a messaging queue.

FIG. 2C illustrates an example of a circular buffer with data segmentsreferenced by addresses stored in a messaging queue and overflow datasegments stored in the messaging queue.

FIG. 3 is a diagram showing an example of a producer process foringesting media content into a server device in preparation fortransmission to a client device on a network.

FIG. 4 is a diagram showing an example of a consumer process foringesting media content into a server device in preparation fortransmission to a client device on a network.

DETAILED DESCRIPTION

The following detailed description is intended to provide severalexamples that will illustrate the broader concepts that are set forthherein, but it is not intended to limit the invention or the applicationand uses of the invention. Furthermore, there is no intention to bebound by any theory presented in the preceding background or thefollowing detailed description.

Various embodiments encode digital media content using a buffer in cachememory and a message queue in main memory. A producer thread storesmedia segments in the buffer and adds pointers to the media segments tothe message queue. The mode of operation in which references are writtento the message queue is referenced herein as copy reference mode.Operating in copy reference mode tends to result in fast ingestion ofmedia into server devices having limited processing power, cachingcapacity, or memory speed.

The buffer may fill, nearly fill, or be at risk of filling as a resultthe producer thread generating media segments faster than messages areconsumed from the message queue. A burst of incoming data, for example,can cause the producer thread to fill the buffer with media segmentsfaster than a consumer thread consumes media segments. The message queuemay copy media segments instead of references into the message queue tomitigate a data burst or a filling buffer. The mode of operation inwhich media segments are copied directly to the message queue isreferred to herein as copy data mode. Switching to copy data mode tendsto prevent data loss when the buffer is operating at or near fullcapacity. Switching between copy reference mode and copy data mode asconditions warrant can greatly improve the performance of the digitalencoder.

In the embodiment illustrated in FIG. 1 , a client device 102 contacts aserver device 120 to establish a connection 135 via network 137. Theserver device 120 may provide media streams to the client 102, such astime and/or place shifted video, video on demand, other digital mediacontent, and/or the like. In the illustrated example, client-side logic115 executed by client device 102 suitably formats secure hypertexttransfer protocol (HTTPS) or similar requests for content. Rather thantransmitting the formatted requests directly on network 137, the clientdevice 102 provides the transmission control protocol (TCP) formattedcontent to a proxy server 111 that operates on the same device 102 asthe TCP client 116. In various embodiments, the TCP client 116 and proxyserver 111 exchange messages using a “dummy” address (e.g.,“client1.dummy.com” in the example of FIG. 1 ) that may not be directlyroutable on network 137. The proxy server 111 receives the TCP-formattedrequests and encapsulates the packets within a secure user datagramprotocol (UDP) type transport layer for transmission to server device120 via network 137. The connectionless packets may use conventionalinternet protocol (IP) addresses or the like that identify client device102 and server device 120 on network 137, as appropriate.

The server device 120 suitably receives the encapsulated request vianetwork 137. A proxy client 141 executed by the server device 120suitably extracts the previously encapsulated TCP/HTTPS content from thesecure UDP-type packets, re-assembles the TCP/HTTPS content as needed,and provides the extracted content to the hypertext transfer protocol(HTTP) server 145 for authentication, authorization, decryption, and/orother processing as needed.

In various embodiments, server device 120 formats the video content tobe delivered in HTTPS structures that are designed for delivery viareliable, connection-based transport connection protocols (TCP). AnHTTPS server 145 operating at the server device 120 suitably formatscontent for connection-based delivery and then supplies the formattedcontent to a proxy client 141 operated by server device 120 thatencapsulates the TCP content for delivery over network 137 via UDP orthe like. The client device 102, conversely, requests and receivessecure content via network 137 at a proxy TCP server that deliverscontent to an HTTP client 115, as appropriate.

FIG. 1 shows an example of a system 100 in which a client device 102obtains streaming content from a server 120 via a wide area network 137.The server device 120 includes a proxy client 141 that receives securecontent from a TCP server 145 and embeds the secure content intoconnectionless transport layer structures for transmission acrossnetwork 137. The secure UDP-like structures may appear as UDP packets tonetwork routers and the like but can be modified to include additionaldata representing additional port numbers and/or packet trackinginformation, as described above.

As noted above, addressing used for HTTPS communications can beabstracted by using a separate uniform resource locator (URL) or otheraddress for the embedded secure data. The TCP server 146 in serverdevice 120 and the TCP client 116 in client device 102 may use aseparate domain name (e.g., “dummy.com”, “commondomain.com”,“dishboxes.com”, etc.) for intra-application communications, if desired.Various embodiments could further use local addresses as sub-domains,e.g., in the form of “local-ip.commondomain.com” or the like, forfurther convenience. The shared domain can be registered with a networkinformation center or the like for cryptographic key management andverification, if desired. Intra-device communications between TCP server146 and TCP client 142 in server device 120, then, will typically use adifferent address than communications between client device 102 andserver device 120. Similarly, TCP client 116 and TCP server 113 inclient device 102 will generally use separate addresses, as appropriate.In some implementations, the “dummy” or proxy addresses can be used toabstract the UDP-type encapsulation from the TCP client 116 and TCPserver 146. That is, these modules may simply communicate with eachother using the “proxy” addresses without regard to the actual IP orother network addresses used on network 137, if desired.

Network 137 is any wide area network (WAN) such as the Internet, atelephony network, a public or private network of any sort, or the like.Network 137 may be based upon TCP/IP protocols, or any other protocolsas desired, including any protocols subsequently developed. Equivalentembodiments may provide device location and/or streaming via local areanetworks, as appropriate.

Server device 120 is any sort of network device having conventionalhardware 123 such as a processor 124, memory 125, and input/outputinterfaces 126 (e.g., a network interface), and any operating system 128to support a media server application 140 having various processingroutes and modules. In the example illustrated in FIG. 1 , server device120 suitably executes a media server application 140 that includes atleast two portions corresponding to an HTTPS server 145 and a proxyclient 141, as described herein. Server application 140 may also includemedia encoder 148 for retrieving and/or encoding video segments, as wellas routines for key management, user/client authentication, and thelike.

In some embodiments, server device 120 uses buffer 130 to receive mediacontent from media source 150 and also to encode media content fortransmission to client device 102 over network 137. Buffer 130 is storedin fast memory having limited capacity, if needed. For example, buffer130 may be stored in cache level memory on a device with limitedcomputing power. Buffer 130 may be one or more circular buffers.

Buffer 130 may support multi-bitrate streaming in various embodiments.For example, buffer 130 may include four circular buffers, with eachcircular buffer of buffer 130 storing data segments (also referred toherein as segments) for a different quality of streaming video.Multi-bitrate streaming is described in greater detail in U.S. Pat. Nos.8,868,772 and 9,998,516, which are incorporated by reference for anypurpose.

Segments written to buffer 130 by a producer thread running on processor124 may be consumed during encoding by a consumer thread running onprocessor 124 in some embodiments. The producer thread writing segmentsto buffer 130 communicates with the consumer thread using a messagequeue 132, function calls, and/or callbacks. In copy reference mode, theproducer thread writes data segments to buffer 130 and may also write anaddress or reference to data segments stored in buffer 130 to themessage queue. The producer thread running on server device 120generally operates in a copy reference mode when writing addresses orreferences to message queue 132. In the example of FIG. 1 , the producerthread is media encoder 148 and the consumer thread is HTTP server 145,although other embodiments could use other producer or consumer threads.

The consumer thread consumes messages from message queue 132. Themessages may include addresses or references in response to serverdevice 120 operating in a copy reference mode. Copy reference mode tendsto reduce the computing resources used by server device 120 by reducingthe number of memory copy operations used by server device 120. Theconservation of computing resources stems from server device 120 storinga reference or pointer to the location of a media segment in cache whenoperating in copy reference mode. Storing a reference to a media segmentin memory tends to be less resource intensive than copying the samemedia segment into memory.

The consumer thread reads the segment identified by the reference storedin message queue 132 from buffer 130. Server device 120 may thus avoidresource-intensive memory copy operations that would otherwise be usedto make copies of data segments for later retrieval by the consumerthread. In the example of FIG. 1 , HTTP server 145 acts as the consumerthread to buffer 130, although other embodiments could use differentconsumer threads.

In various embodiments, server device 120 encounters conditions causingthe producer thread to write segments to buffer 130 and addresses tomessage queue 132 faster than the consumer thread consumes addressesfrom message queue 132 and reads the identified segments from buffer130. For example, a burst of data from media source 150 may result inthe producer thread filling buffer 130 faster than the consumer threadcan empty buffer 130, thereby creating stalls in processing, data loss,or other undesirable conditions. To prevent this, server device 120switches into a copy data mode when an overflow or burst condition onbuffer 130 or message queue 132 is detected.

In embodiments suitable for multi-bitrate streaming, server device 120may divide buffer 130 and message queue 132, so that different parts ofbuffer 130 and message queue 132 are assigned to each of the differentbitrate streams. Server device 120 may include multiple buffers 130 andmultiple message queues 132 in such embodiments. For example, buffer 130may be divided into four buffers having similar or equal length formulti-bitrate streaming content that processes four different bitrates.Buffer length may represent the number of data segments from a bitratestream that fit into a buffer. Continuing the foregoing example, buffer130 may have a total buffer length of 16 divided among four differentbitrate streams, with each stream having a buffer of length 4. Otherbuffer capacities and lengths may be used depending on system capacity,video quality, or other system characteristics.

In various embodiments, server application 140 obtains media contentfrom a media source 150. Media source 150 may be a television tuner or astorage device that is formatted to include a database of media content,although other embodiments may operate in any other manner or receivecontent from any other source. In the example illustrated in FIG. 1 ,the various functions and modules of server application 140 areimplemented using software or firmware logic that is stored in memory125 for execution by processor 124, although other embodiments could useequivalent computing hardware to perform the various functions, asdesired.

Examples of different types of server devices 120 could include anystreaming media source such as a file server or content delivery network(CDN), a video game device, a time and/or placeshifting device, and/orthe like. In some implementations, server device 120 is a home-typeserver such as a local storage digital video recorder (LSDVR),placeshifting device, and/or other media server device. One example of aserver device 120 used in some implementations could be the AirTVClassic device that is available from http://www.airtv.net, althoughequivalent embodiments could be used with any number of other DVRs,media receivers/players, video on demand (VOD) servers, set top boxes,video game consoles, time or place shifting devices, and/or the like.U.S. Pat. No. 7,795,062 provides additional detail about severalexamples of place shifting devices and techniques. Equivalent conceptscould be implemented in any number of other devices or systems.

Media server application 140 may be designed and implemented in anymanner. In the example shown in FIG. 1 , server application 140 suitablyincludes a media encoder 148, an HTTP server 145, and a local proxyclient 141 as described herein. The media encoder 148 obtains contentfrom a database or other media source 150, as desired. Content may beformatted into adaptive streaming segments such as those described inU.S. Pat. Publication No. 2020/0389510 or the like, and otherembodiments may use different encoding and delivery schemes.

As shown in FIG. 1 , HTTP server 145 provides authentication,authorization, and encryption/decryption in accordance with HTTP/HTTPSprotocols. HTTP server 145 consumes content from buffer 130 usingreferences in message queue 132 to access the referenced location inbuffer 130 storing the content. Pointers, counters, index values, orother techniques may be used indicate content has been consumed from areferenced location in buffer 130 in response to operation in copyreference mode.

FIG. 1 also shows a TCP server 146. In practice, higher level HTTP/HTTPScontent can be encapsulated within TCP frames for transmission over aninternet protocol network such as the internet. In this example,however, TCP packets are provided within media server application 140 toTCP client 142 operating as part of local proxy client 141. The TCPclient 142 handles interactions with the TCP/HTTP server 145, therebyabstracting network 137 from the server 145. As noted above, receivedTCP packets are appropriately encapsulated within connectionlessUDP-type transport layer frames by the UDP server 143. In the example ofFIG. 1 , UDP server 143 is indicated as a “SUDP” server for “secureUDP”, “Sling UDP”, “specialized UDP”, “supplemented UDP” or the like. Invarious embodiments, TCP packets received by TCP client 142 are storedin a buffer 144 or the like until they can be processed by UDP server143. Buffer 144 may make use of memory 125 and/or any other data storageavailable to server device 120, as appropriate. Buffer 144 may alsostore packets moving in the opposite direction. That is, connectionlesspackets received via network 137 may be stripped of their UDP-typeencapsulation and stored in buffer 144 prior to forwarding by TCP client142, as desired.

Client device 102 is any device capable of communicating on network 137to obtain data or services from server 120. In various embodiments,client device 102 is a mobile phone, tablet, computer, and/or the likethat interfaces with network 137 via hardware (e.g., processors, memory,input interfaces, and the like) and any operating system 108 capable ofsupporting a client application 110. Client application 110 typicallyincludes at least two portions corresponding to a proxy server 111 andan HTTP client 115, as described herein. Client application 110 mayfurther include additional logic 118 for media decoding, sequencing,rendering, and/or the like. In various embodiments, client application110 and its various components are implemented using software orfirmware logic that is stored in memory 105 for execution by processor104. Equivalent embodiments could use other computing structures andsystems to implement similar features, as desired.

As described above, proxy server 111 suitably includes an SUDP client112 that receives connectionless packets from server 120 via network 137and TCP server 113 that serves de-encapsulated TCP/HTTP packets to HTTPclient 115. Proxy server 111 may also include a buffer 114 that allowsfor temporary storage of packets exchanged between SUDP client 112 andTCP server 113. Buffer 114 may use memory 105 or other data storageavailable to client device 102, as desired.

HTTP client 115 performs HTTP/HTTPS encoding, including processing ofdigital certificates, encryption, and the like. As noted above withregard to HTTP server 145, HTTP constructs may be transported using aTCP stack prior to encapsulation within connectionless UDP-type frames,as desired. In some implementations, the local addressing used betweenHTTP client 115 and HTTP server 145 could be compatible such that theUDP encapsulation and transmission on network 137 are transparent to theHTTP client 115 and HTTP server 145, as desired. Again, variousembodiments could use any protocols, as desired.

FIGS. 2A through 2C illustrate examples of buffer 130 and message queue132 as stored in memory 125, in accordance with various embodiments.Memory 125 may include various physical levels of memory 125 accessibleby processor 124 such as, in descending order of read/write speeds whenaccessed by processor 124, registers, cache, main memory, and permanentstorage, for example. In some embodiments, buffer 130 may be stored incache to improve system performance when reading or writing from datastored in buffer 130. Data segments 201 through 212 shown in FIGS. 2 maytherefore be stored in buffer 130. The data segments may include chunksof media generated from a media stream or media file received from mediasource 150.

Server device 120 may have a limited amount of cache available forbuffer 130 such that buffer 130 has a maximum storage capacity, alsoreferred to as a maximum length. The maximum length of buffer 130 may beequal to the maximum number of data segments that can be stored inbuffer 130. For example, if buffer 130 had a maximum storage capacity of16 megabytes (MB) and each data segment was 0.96 MB, then the maximumbuffer length would be 16 data segments. In another example, if cachehad a storage capacity of 16 MB, each data segment was 0.96 MB, andcache held four instances of buffer 130 having an equal maximum length,then the maximum buffer length would be four data segments. Otherembodiments could use any number of different buffers, or buffers havingequal or unequal lengths.

Server device 120 may use the main memory level of memory 125 to storemessage queue 132. Message queue 132 is configured to store addresses221 - 232, or other pointers that identify the location of data segmentsstored in buffer 130. Message queue 132 may also be configured to storecopies of data segments in response to buffer 130 being full.

With reference to FIG. 2A, buffer 130 and message queue 132 are shownoperating in a copy reference mode according to one embodiment. In copyreference mode, message queue 132 is storing addresses 221 - 227 thatpoint to segments 201 - 207, respectively. For example, message queue132 stores address 221 referencing segment 201, which is stored inbuffer 130. A consumer thread running on processor 124 (of FIG. 1 )reads address 221 and uses address 221 to access the referenced locationand to read segment 201 directly from buffer 130.

FIG. 2B shows an example embodiment with buffer 130 full of segments 201through 212 and message queue 132 storing addresses 221 through 232.Each address stored in message queue 132 points to the location inbuffer 130 of a segment. In the example embodiments of FIG. 2B, buffer130 has a maximum length of twelve since it can hold at most twelvesegments in this example. Message queue 132 has 12 addresses, with eachaddress 221 through 232 referencing a segment 201 through 212. Thenumber of addresses stored in message queue 132 is equal to the maximumlength of buffer 130, leaving no additional space in buffer 130.

When buffer 130 fills, server device 120 (of FIG. 1 ) switches to a copydata mode. The switching condition to change from copy reference mode tocopy data mode may be, for example, when the number of addresses storedin a message queue meets or exceeds a threshold value, generallyrelating to the length, fullness, or utilization of buffer 130.Continuing the example, server device 120 may switch to copy data modein response to the number of addresses stored in message queue 132threatening to exceed the maximum length of buffer 130. The specificthreshold value may be based on the maximum buffer length, the maximumnumber of addresses stored in message queue 132, or any other conditionsuitable for preventing server device 120 from overwriting data inbuffer 130 before the data is consumed. Various embodiments may set thethreshold value to be slightly less than the full capacity of thebuffer, if desired.

Referring now to FIG. 2C, buffer 130 and message queue 132 are shownoperating in a copy data mode, in accordance with an example embodiment.Server device 120 (of FIG. 1 ) operating in copy data mode writes datasegments directly to message queue 132. Message queue 132 may containaddresses 221 through 232 along with segments 213 through 215. Copyingsegments 213, 214, and 215 to message queue 132 is typically slower thanwriting the segments to faster buffer 130, but this tends to avoidoverwriting unconsumed segments 201 through 212 stored in buffer 130.The slower processing may also give the consumer thread time to processthe backlog of data stored in buffer 130.

Server device 120 continues to operate in copy data mode until theproducer thread or consumer thread detect a switching condition. Serverdevice 120 switches to a copy reference mode in response to detectingthe switching condition. The switching condition may indicate thatbuffer 130 is no longer full of segments. For example, server device 120may switch to a copy reference mode in response to detecting that thenumber of addresses stored in the message queue is less than (or equalto) a second threshold value. The second threshold value may be set toindicate buffer utilization permits return to copy reference mode. Forexample, a suitable threshold value might be the constant 1. A constantof 1 would indicate that either 1 or zero addresses are stored inmessage queue 132. Other suitable threshold values may be calculatedusing maximum buffer length minus a positive, nonzero constant.Threshold values may also be dynamically calculated depending on theoperational constraints and hardware capacity of server device 120.While several examples of threshold values are given herein, thesevalues are intended as examples only and are not limiting.

FIG. 3 shows an example process 300 for encoding media content on serverdevice 120 using buffer 130 and message queue 132 for transmission toclient device 102. The various functions described in FIG. 3 may beperformed by programmed logic (e.g., software or firmware) stored withinmemory 105 or 125 and executed by processors 104 or 124, as appropriate.Other embodiments may perform additional functions, and/or may organizethe different functions in an equivalent but alternate manner.

Server device 120 may receive media content from media source 150 (Block302). Media content may be media streams or media files. Media contentmay include live streams received from media source 150 such as anetwork broadcast. Media content may also include prerecorded or ondemand content.

Server device 120 may generate segments from the media content (Block304). Segments may include data segments of the media content andmetadata. The segments may be multi-bitrate segments encoded at variousbitrates to support different levels of video or audio quality on clientdevice 102 based on bandwidth across network 137. The producer threadmay initially write data segments to buffer 130 while operating in acopy reference mode.

Server device 120 may check whether a switching condition is met (Block306). For example, server device 120 may check whether the number ofaddresses stored in a message queue 132 (MQ) is greater than or equal toa threshold value T₁. The threshold value T₁ may be based on the maximumlength of buffer 130. The threshold value T₁ may be a constant value insome embodiments. The maximum length of buffer 130 may be predeterminedand fixed, or it may be adapted in operation as desired. The maximumbuffer length may be dynamically calculated based on the amount ofstorage allocated to buffer 130. For example, a buffer with 16 kilobytes(KB) of capacity storing media segments of 1 KB would have a maximumbuffer length of 16. Continuing the example, if the buffer’s storagecapacity were reduced to 4 KB, the maximum length of the buffer would be4.

Server device 120 may write the address of a data segment in buffer 130to message queue 132 (Block 308) in response to message queue 132containing fewer addresses than the maximum length of buffer 130. Statedanother way, server device 120 may remain in copy reference mode inresponse to message queue 132 containing fewer addresses than themaximum length of buffer 130. Server device 120 may remain in copyreference mode until a switching condition is detected to indicate thebuffer is nearing its capacity.

Server device 120 may write data segment to the message queue inresponse to the number of addresses stored in the message queue beinggreater than or equal to the maximum buffer length (Block 310). Statedanother way, server device 120 may switch to copy data mode in responseto message queue 132 containing a number of addresses equal to orgreater than the maximum length of buffer 130. Server device 120 maywrite copies of data segments to message queue 132 in response to buffer130 being full. The data segments may be multi-bitrate segments.

Server device 120 may switch from copy data mode to copy reference modein response to detecting a second switching condition (Block 312). Forexample, server device 120 may check whether the number of addressesstored in the message queue 132 is less than (or equal to) a thresholdvalue T₂. Server device 120 may also check whether the number ofaddresses stored in the message queue 132 greater than (or equal to) thethreshold value T₂. The threshold value T₂ may be less than the maximumlength of buffer 130. For example, the threshold value T₂ may be 1.Addresses referencing data segments stored in buffer 130 may be writtento message queue 132 in response to server device 120 being in copyreference mode.

Server device 120 may be an embedded device having limited computingresources available. The use of buffer 130 and message queue 132 asdescribed herein may enable such devices to encode media streams despitelimited computing resources. The use of buffer 130 and message queue 132as described herein may enable server device 120 to ingest and encodestreams of live content or other media content susceptible to burstoutput or similar conditions in which server device 120 ingests mediacontent at a rate greater than the encoded content is consumed. Buffer130 and message queue 132 may tend to reduce processor and memory costsby reducing the number of memory copy commands used to encode mediastreams.

Processor 124 of server device 120 may run process 300 on a producerthread. A consumer thread may run in parallel to consume data written tomessage queue 132. A single threaded processor may support both threadsby context switching between separate processes for the consumer threadand the producer thread. In the example illustrated in FIG. 1 ,operating system 128 may switch context between media encoder 148 (theproducer thread) and HTTP server 145 (the consumer thread). The producerthread may variously write references or data segments to buffer 130 andmessage queue 132 as described above. The producer may write an addressto message queue 132 without writing a copy of the data segment tomessage queue 132. By avoiding a copy command, the producer may tend tooptimize CPU and preserve memory bandwidth.

The consumer thread may consume the data from message queue 132. Serverdevice 120 may include a hard condition that the queue length ofaddresses stored in message queue 132 should not exceed the maximumlength of buffer 130 to avoid writing over the data which has not beenconsumed yet. Server device 120 may detect such scenarios through acallback (e.g., for burst output), by measuring the length of messagequeue 132, or using an index pointer that tracks the next messagelocation. Server device 120 may switch to copy data mode in response todetecting a scenario where buffer data would be overwritten. In copydata mode, an actual copy of data is written to the message queueinstead of writing a reference to the data, so that data will not belost even if the length of message queue 132 exceeds the maximum bufferlength.

FIG. 4 shows an example process 400 for execution by a consumer threadrunning on server device 120 using buffer 130 and message queue 132. Thevarious functions described in FIG. 4 may be performed by programmedlogic (e.g., software or firmware) stored within memory 105 or 125 andexecuted by processors 104 or 124, as appropriate. Other embodiments mayperform additional functions, and/or may organize the differentfunctions in an equivalent but alternate manner.

The consumer thread running on server device 120 reads the next messagefrom message queue 132 (Block 402). The message body may be a referenceto a location in buffer 130 or a data segment. The message can includevarious types of metadata such as, for example, metadata indicatingwhere in memory the message begins and ends. Metadata may also include aflag indicating the message is a reference or a data segment. Themessage can include an address referencing to a location in messagequeue 132 where the next message begins. Other types of metadata may beincluded in the message for use in determining whether the message bodyis a reference or a segment.

The consumer thread checks whether the message is a reference (Block404). The consumer thread may use the body of the message or metadata todetermine whether the message is a reference or a data segment. Theconsumer thread may also use callbacks or function calls to communicatewith the producer thread and determine whether a message is a referenceor a data segment.

If the message is a reference, then the consumer thread reads a datasegment from the location in buffer 130 referenced by the message (Block406). If the message is a data segment, then the consumer thread readsthe data segment directly from message queue 132 (Block 408). Readingand writing data segments in buffer 130 stored in cache tends to befaster than reading and writing data segments in message queue 132stored in main memory.

System 100 using buffer 130 and message queue 132 as described hereinmay seamlessly handle data streaming when the consumer temporarily lagsbehind the producer due to asynchronous operation. The system may alsohandle burst output, where data comes in faster than real time and morethan the maximum buffer length of segments are generated. System 100 mayswitch to reference mode as soon as possible to conserve processing andmemory resources, thus tending to minimize bandwidth constraints.

The term “exemplary” is used herein to represent one example, instance,or illustration that may have any number of alternates. Anyimplementation described herein as “exemplary” should not necessarily beconstrued as preferred or advantageous over other implementations. Whileseveral exemplary embodiments have been presented in the foregoingdetailed description, it should be appreciated that a vast number ofalternate but equivalent variations exist, and the examples presentedherein are not intended to limit the scope, applicability, orconfiguration of the invention in any way. To the contrary, variouschanges may be made in the function and arrangement of the variousfeatures described herein without departing from the scope of the claimsand their legal equivalents.

What is claimed is:
 1. An automated process comprising: receiving media content from a media source; generating a plurality of segments from the media content; writing, to a buffer of a computing device, a first segment from the plurality of segments; writing, to a message queue of the computing device, a first address referencing the first segment of the media content stored in the buffer; switching operating modes in response to a number of addresses stored in the message queue being greater than or equal to a first threshold value; and writing a second segment of the media content to the message queue of the computing device.
 2. The automated process of claim 1, wherein the first threshold value is equal to a maximum length of the buffer.
 3. The automated process of claim 2, wherein the maximum length of the buffer is equal to a maximum number of data segments that fit into the buffer.
 4. The automated process of claim 2, wherein the plurality of segments comprises multi-bitrate segments.
 5. The automated process of claim 4, wherein the buffer is stored in a cache-level memory of the computing device, and wherein the message queue is in a main memory of the computing device, the main memory having slower access speeds than the cache-level memory.
 6. The automated process of claim 2, further comprising writing, to the message queue, a second address referencing a third segment of the media content stored in the buffer in response to the number of addresses stored in the message queue being less than or equal to a second threshold value.
 7. An automated process executed by a processor of a server device to ingest media content, the automated process comprising: receiving the media content; generating a plurality of segments of the media content; writing, to a message queue, a reference to a first segment from the segments stored in a buffer in response to the server device operating in a copy reference mode; and switching into a copy data mode to write a second segment from the segments to the message queue in response to detecting a first switching condition of the message queue.
 8. The automated process of claim 7, further comprising writing, to the message queue, a second segment from the segments stored in the buffer in response to the server device operating in the copy data mode.
 9. The automated process of claim 7, further comprising writing, to the message queue, a second reference to a third segment of the media content stored in the buffer in response to a number of references stored in the message queue being less than or equal to a threshold value.
 10. The automated process of claim 9, wherein the threshold value is a maximum length of the buffer.
 11. The automated process of claim 7, wherein the segments comprise multi-bitrate segments.
 12. The automated process of claim 7, wherein the buffer is stored in a cache-level memory of the server device, and wherein the message queue is in a main memory of the server device, the main memory having slower read and write speeds than the cache-level memory.
 13. The automated process of claim 7, wherein detecting the first switching condition of the message queue comprises determining a number of references stored in the message queue is greater than or equal to a threshold value.
 14. The automated process of claim 7, wherein the server device consumes the reference to the first segment from the message queue and reads the first segment from the buffer to encode the media content.
 15. A server device comprising a processor, a non-transitory data storage, and an interface to a network, wherein the non-transitory data storage is configured to store computer-executable instructions that, when executed by the processor, perform an automated process to ingest media content, the automated process comprising: receiving the media content from a media source; generating a plurality of segments from the media content; writing, to a message queue, a first address referencing a first segment of the media content stored in a buffer; and writing, to the message queue, a second segment of the media content in response to a number of addresses stored in the message queue being greater than or equal to a first threshold value.
 16. The server device of claim 15, wherein the server device switches into a copy data mode in response to the number of addresses stored in the message queue being greater than or equal to the first threshold value.
 17. The server device of claim 15, wherein the automated process further comprises switching, by the processor, into a copy reference mode in response to the number of addresses stored in the message queue being less than or equal to a second threshold value.
 18. The server device of claim 17, wherein the automated process further comprises writing, by the processor and to the message queue, a third address referencing a third segment of the media content in response to the server device being in the copy reference mode.
 19. The server device of claim 15, wherein the buffer is stored in a cache-level memory of the server device, and wherein the message queue is stored in a main memory of the server device, the main memory having slower access speeds than the cache-level memory.
 20. The server device of claim 15, wherein the message queue comprises a first-in-first-out queue. 